Object-based multimedia content description schemes and applications for MPEG-7

In this paper, we describe description schemes (DSs) for image, video, multimedia, home media, and archive content proposed to the MPEG-7 standard. MPEG-7 aims to create a multimedia content description standard in order to facilitate various multimedia searching and "ltering applications. During the design process, special care was taken to provide simple but powerful structures that represent generic multimedia data. We use the extensible markup language (XML) to illustrate and exemplify the proposed DSs because of its interoperability and #exibility advantages. The main components of the image, video, and multimedia description schemes are object, feature classi"cation, object hierarchy, entity-relation graph, code downloading, multi-abstraction levels, and modality transcoding. The home media description instantiates the former DSs proposing the 6-W semantic features for objects, and 1-P physical and 6-W semantic object hierarchies. The archive description scheme aims to describe collections of multimedia documents, whereas the former DSs only aim at individual multimedia documents. In the archive description scheme, the content of an archive is represented using multiple hierarchies of clusters, which may be related by entity-relation graphs. The hierarchy is a speci"c case of entity-relation graph using a containment relation. We explicitly include the hierarchy structure in our DSs because it is a natural way of de"ning composite objects, a more e$cient structure for retrieval, and the representation structure used in MPEG-4. We demonstrate the feasibility and the e$ciency of our description schemes by presenting applications that already use the proposed structures or will greatly bene"t from their use. These applications are the visual apprentice, the AMOS-search system, a multimedia broadcast news browser, a storytelling system, and an image meta-search engine, MetaSEEk. ( 2000 Elsevier Science B.V. All rights reserved.